Introducing Hierarchy in Energy Games

In this work we introduce hierarchy in wireless networks that can be modeled by a decentralized multiple access channel and for which energy-efficiency is the main performance index. In these networks users are free to choose their power control strategy to selfishly maximize their energy-efficiency. Specifically, we introduce hierarchy in two different ways: 1. Assuming single-user decoding at the receiver, we investigate a Stackelberg formulation of the game where one user is the leader whereas the other users are assumed to be able to react to the leader's decisions; 2. Assuming neither leader nor followers among the users, we introduce hierarchy by assuming successive interference cancellation at the receiver. It is shown that introducing a certain degree of hierarchy in non-cooperative power control games not only improves the individual energy efficiency of all the users but can also be a way of insuring the existence of a non-saturated equilibrium and reaching a desired trade-off between the global network performance at the equilibrium and the requested amount of signaling. In this respect, the way of measuring the global performance of an energy-efficient network is shown to be a critical issue.Comment: Accepted for publication in IEEE Trans. on Wireless Communication

Dynamic capacity provision for wireless sensors connectivity: A profit optimization approach

[EN] We model a wireless sensors' connectivity scenario mathematically and analyze it using capacity provision mechanisms, with the objective of maximizing the profits of a network operator. The scenario has several sensors' clusters with each one having one sink node, which uploads the sensing data gathered in the cluster through the wireless connectivity of a network operator. The scenario is analyzed both as a static game and as a dynamic game, each one with two stages, using game theory. The sinks' behavior is characterized with a utility function related to the mean service time and the price paid to the operator for the service. The objective of the operator is to maximize its profits by optimizing the network capacity. In the static game, the sinks' subscription decision is modeled using a population game. In the dynamic game, the sinks' behavior is modeled using an evolutionary game and the replicator dynamic, while the operator optimal capacity is obtained solving an optimal control problem. The scenario is shown feasible from an economic point of view. In addition, the dynamic capacity provision optimization is shown as a valid mechanism for maximizing the operator profits, as well as a useful tool to analyze evolving scenarios. Finally, the dynamic analysis opens the possibility to study more complex scenarios using the differential game extension.The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Spanish Ministry of Economy and Competitiveness through project TIN2013-47272-C2-1-R; AEI/FEDER, UE through project TEC2017-85830-C2-1-P; and co-supported by the European Social Fund BES-2014-068998.Sanchis-Cano, Á.; Guijarro, L.; Condoluci, M. (2018). Dynamic capacity provision for wireless sensors connectivity: A profit optimization approach. International Journal of Distributed Sensor Networks (Online). 14(4):1-14. https://doi.org/10.1177/1550147718772544S114144Weiser, M. (1991). The Computer for the 21st Century. Scientific American, 265(3), 94-104. doi:10.1038/scientificamerican0991-94Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M. (2013). Internet of Things (IoT): A vision, architectural elements, and future directions. Future Generation Computer Systems, 29(7), 1645-1660. doi:10.1016/j.future.2013.01.010Perera, C., Zaslavsky, A., Christen, P., & Georgakopoulos, D. (2013). Sensing as a service model for smart cities supported by Internet of Things. Transactions on Emerging Telecommunications Technologies, 25(1), 81-93. doi:10.1002/ett.2704Wang, N., Hossain, E., & Bhargava, V. K. (2016). Joint Downlink Cell Association and Bandwidth Allocation for Wireless Backhauling in Two-Tier HetNets With Large-Scale Antenna Arrays. IEEE Transactions on Wireless Communications, 15(5), 3251-3268. doi:10.1109/twc.2016.2519401Chowdhury, M. Z., Jang, Y. M., & Haas, Z. J. (2013). Call admission control based on adaptive bandwidth allocation for wireless networks. Journal of Communications and Networks, 15(1), 15-24. doi:10.1109/jcn.2013.000005Nan, G., Mao, Z., Yu, M., Li, M., Wang, H., & Zhang, Y. (2014). Stackelberg Game for Bandwidth Allocation in Cloud-Based Wireless Live-Streaming Social Networks. IEEE Systems Journal, 8(1), 256-267. doi:10.1109/jsyst.2013.2253420Zhu, K., Niyato, D., Wang, P., & Han, Z. (2012). Dynamic Spectrum Leasing and Service Selection in Spectrum Secondary Market of Cognitive Radio Networks. IEEE Transactions on Wireless Communications, 11(3), 1136-1145. doi:10.1109/twc.2012.010312.110732Vamvakas, P., Tsiropoulou, E. E., & Papavassiliou, S. (2017). Dynamic Provider Selection & Power Resource Management in Competitive Wireless Communication Markets. Mobile Networks and Applications, 23(1), 86-99. doi:10.1007/s11036-017-0885-yNiyato, D., Hoang, D. T., Luong, N. C., Wang, P., Kim, D. I., & Han, Z. (2016). Smart data pricing models for the internet of things: a bundling strategy approach. IEEE Network, 30(2), 18-25. doi:10.1109/mnet.2016.7437020Guijarro, L., Pla, V., Vidal, J. R., & Naldi, M. (2016). Maximum-Profit Two-Sided Pricing in Service Platforms Based on Wireless Sensor Networks. IEEE Wireless Communications Letters, 5(1), 8-11. doi:10.1109/lwc.2015.2487259Romero, J., Guijarro, L., Pla, V., & Vidal, J. R. (2017). Price competition between a macrocell and a small-cell service provider with limited resources and optimal bandwidth user subscription: a game-theoretical model. Telecommunication Systems, 67(2), 195-209. doi:10.1007/s11235-017-0331-2Al Daoud, A., Alanyali, M., & Starobinski, D. (2010). Pricing Strategies for Spectrum Lease in Secondary Markets. IEEE/ACM Transactions on Networking, 18(2), 462-475. doi:10.1109/tnet.2009.2031176Do, C. T., Tran, N. H., Huh, E.-N., Hong, C. S., Niyato, D., & Han, Z. (2016). Dynamics of service selection and provider pricing game in heterogeneous cloud market. Journal of Network and Computer Applications, 69, 152-165. doi:10.1016/j.jnca.2016.04.012Tsiropoulou, E. E., Vamvakas, P., & Papavassiliou, S. (2017). Joint Customized Price and Power Control for Energy-Efficient Multi-Service Wireless Networks via S-Modular Theory. IEEE Transactions on Green Communications and Networking, 1(1), 17-28. doi:10.1109/tgcn.2017.2678207Sanchis-Cano, A., Romero, J., Sacoto-Cabrera, E., & Guijarro, L. (2017). Economic Feasibility of Wireless Sensor Network-Based Service Provision in a Duopoly Setting with a Monopolist Operator. Sensors, 17(12), 2727. doi:10.3390/s17122727Weber, T. A. (2011). Optimal Control Theory with Applications in Economics. doi:10.7551/mitpress/9780262015738.001.0001Mandjes, M. (2003). Pricing strategies under heterogeneous service requirements. Computer Networks, 42(2), 231-249. doi:10.1016/s1389-1286(03)00191-9Shariatmadari, H., Ratasuk, R., Iraji, S., Laya, A., Taleb, T., Jäntti, R., & Ghosh, A. (2015). Machine-type communications: current status and future perspectives toward 5G systems. IEEE Communications Magazine, 53(9), 10-17. doi:10.1109/mcom.2015.7263367Ng, C.-H., & Soong, B.-H. (2008). Queueing Modelling Fundamentals. doi:10.1002/9780470994672Mendelson, H. (1985). Pricing computer services: queueing effects. Communications of the ACM, 28(3), 312-321. doi:10.1145/3166.3171Altman, E., Boulogne, T., El-Azouzi, R., Jiménez, T., & Wynter, L. (2006). A survey on networking games in telecommunications. Computers & Operations Research, 33(2), 286-311. doi:10.1016/j.cor.2004.06.005Belleflamme, P., & Peitz, M. (2015). Industrial Organization. doi:10.1017/cbo9781107707139Reynolds, S. S. (1987). Capacity Investment, Preemption and Commitment in an Infinite Horizon Model. International Economic Review, 28(1), 69. doi:10.2307/2526860Barron, E. N. (2013). Game Theory. doi:10.1002/9781118547168Sandholm, W. (2009). Pairwise Comparison Dynamics and Evolutionary Foundations for Nash Equilibrium. Games, 1(1), 3-17. doi:10.3390/g1010003Schlag, K. H. (1998). Why Imitate, and If So, How? Journal of Economic Theory, 78(1), 130-156. doi:10.1006/jeth.1997.234

Game theory framework for MAC parameter optimization in energy-delay constrained sensor networks

Optimizing energy consumption and end-to-end (e2e) packet delay in energy-constrained, delay-sensitive wireless sensor networks is a conflicting multiobjective optimization problem. We investigate the problem from a game theory perspective, where the two optimization objectives are considered as game players. The cost model of each player is mapped through a generalized optimization framework onto protocol-specific MAC parameters. From the optimization framework, a game is first defined by the Nash bargaining solution (NBS) to assure energy consumption and e2e delay balancing. Secondy, the Kalai-Smorodinsky bargaining solution (KSBS) is used to find an equal proportion of gain between players. Both methods offer a bargaining solution to the duty-cycle MAC protocol under different axioms. As a result, given the two performance requirements (i.e., the maximum latency tolerated by the application and the initial energy budget of nodes), the proposed framework allows to set tunable system parameters to reach a fair equilibrium point that dually minimizes the system latency and energy consumption. For illustration, this formulation is applied to six state-of-the-art wireless sensor network (WSN) MAC protocols: B-MAC, X-MAC, RI-MAC, SMAC, DMAC, and LMAC. The article shows the effectiveness and scalability of such a framework in optimizing protocol parameters that achieve a fair energy-delay performance trade-off under the application requirements

Adaptive Honeypot Engagement through Reinforcement Learning of Semi-Markov Decision Processes

A honeynet is a promising active cyber defense mechanism. It reveals the fundamental Indicators of Compromise (IoCs) by luring attackers to conduct adversarial behaviors in a controlled and monitored environment. The active interaction at the honeynet brings a high reward but also introduces high implementation costs and risks of adversarial honeynet exploitation. In this work, we apply infinite-horizon Semi-Markov Decision Process (SMDP) to characterize a stochastic transition and sojourn time of attackers in the honeynet and quantify the reward-risk trade-off. In particular, we design adaptive long-term engagement policies shown to be risk-averse, cost-effective, and time-efficient. Numerical results have demonstrated that our adaptive engagement policies can quickly attract attackers to the target honeypot and engage them for a sufficiently long period to obtain worthy threat information. Meanwhile, the penetration probability is kept at a low level. The results show that the expected utility is robust against attackers of a large range of persistence and intelligence. Finally, we apply reinforcement learning to the SMDP to solve the curse of modeling. Under a prudent choice of the learning rate and exploration policy, we achieve a quick and robust convergence of the optimal policy and value.Comment: The presentation can be found at https://youtu.be/GPKT3uJtXqk. arXiv admin note: text overlap with arXiv:1907.0139

Strategic Learning for Active, Adaptive, and Autonomous Cyber Defense

The increasing instances of advanced attacks call for a new defense paradigm that is active, autonomous, and adaptive, named as the \texttt{`3A'} defense paradigm. This chapter introduces three defense schemes that actively interact with attackers to increase the attack cost and gather threat information, i.e., defensive deception for detection and counter-deception, feedback-driven Moving Target Defense (MTD), and adaptive honeypot engagement. Due to the cyber deception, external noise, and the absent knowledge of the other players' behaviors and goals, these schemes possess three progressive levels of information restrictions, i.e., from the parameter uncertainty, the payoff uncertainty, to the environmental uncertainty. To estimate the unknown and reduce uncertainty, we adopt three different strategic learning schemes that fit the associated information restrictions. All three learning schemes share the same feedback structure of sensation, estimation, and actions so that the most rewarding policies get reinforced and converge to the optimal ones in autonomous and adaptive fashions. This work aims to shed lights on proactive defense strategies, lay a solid foundation for strategic learning under incomplete information, and quantify the tradeoff between the security and costs.Comment: arXiv admin note: text overlap with arXiv:1906.1218

Markov Decision Evolutionary Game for Energy Management in Delay Tolerant Networks

In this paper, we apply the concepts of Markov decision evolutionary games to non-cooperative forwarding control of Delay Tolerant Networks (DTN). Specifically, we rely on the design of mechanisms at the source node to study forwarding probability of the message in a DTN using the two-hop routing. We study the forwarding probability as a function of the competition within a large population of mobiles which need occasionally to make some action. In particular, for each message generated by a source, each mobile may take a decision that concerns the strategy by which the mobile participates to the relaying of the message from source to destination. A mobile that participates receives a unit of reward if it is the first to deliver a copy of the packet to the destination. The action taken by a mobile determine not only the immediate reward but also the transition probability to its next battery energy state. We characterize the Evolutionary Stable Strategies (ESS) for these games and propose a method to compute them. We also propose a mechanism design at the source in order to maximize the message delivery probability to the destination, given the equilibrium behavior (called Evolutionary Stable Strategy - ESS)

A New Way of Thinking Utility in Pricing Mechanisms: A Neural Network Approach

Pricing is regarded as a solution to congestion control in telecommunication networks. Most mathematical models involve a so-called utility function accounting for the users ’ willingness to pay. However, this utility function is unknown in practice in terms of shape and important arguments. We propose here to limit this degree of uncertainty by aggregating all arguments in one quantity, the perceived quality of service, estimated using a Random Neural Network as a statistical learning tool according to the PSQA method. After arguing for this approach, we present a way of applying this tool to a model with two types of traffic and two classes of customers using strict priorities. We illustrate the proposal using a specific simple case

Designing virus-resistant networks: A game-formation approach

Forming, in a decentralized fashion, an optimal network topology while balancing multiple, possibly conflicting objectives like cost, high performance, security and resiliency to viruses is a challenging endeavor. In this paper, we take a game-formation approach to network design where each player, for instance an autonomous system in the Internet, aims to collectively minimize the cost of installing links, of protecting against viruses, and of assuring connectivity. In the game, minimizing virus risk as well as connectivity costs results in sparse graphs. We show that the Nash Equilibria are trees that, according to the Price of Anarchy (PoA), are close to the global optimum, while the worst-case Nash Equilibrium and the global optimum may significantly differ for small infection rate and link installation cost. Moreover, the types of trees, in both the Nash Equilibria and the optimal solution, depend on the virus infection rate, which provides new insights into how viruses spread: for high infection rate tau , the path graph is the worst- and the star graph is the best-case Nash Equilibrium. However, for small and intermediate values of tau , trees different from the path and star graphs may be optimal.Network Architectures and ServicesElectrical Engineering, Mathematics and Computer Scienc